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ABSTBACT 

Trying to reconstruct his role in the evaluation of 
interdisciplinary, problem-oriented courses at Cornell University's 
Center fo*. International studies (CIS), one 

educationist/administrator focused on the evaluation of questions. 
Primary staff goals included the demonstration that CIS 
interdisciplinary courses would better serve students as an 
introduction to represented disciplines than standard courses and CIS 
would show students how knowledge is organized into disciplines whose 
interactions can help illuminate international problems. Problems 
related to the collaborative nature of the evaluation of CIS courses 
were examined in detail. (Author/BJG) 



IMPACTS ON AN EDUCATIONIST/ADMINISTRATOR 
Stephen C. Brock 
. Cornell University 



Each of us who participated in' the study of the CentSr for 
International Studies interdisciplinary, problem-oriented courses sets 
out, here, to reconstruct a reality of earliest expectations, motives 
and orientations. We do this in order to select and present events and 
the consequences of events which have most contemporary importance to 
us. For me, the most important effect cf this evaluation study gua 
evaluation studies has been what I have learned of the evolution of the 
kinds of questions which a study =first asks and seeks to answer. 

Therefore, I begin by recollecting the content of meetings in 
the Fall of 1971 which brought m6 together wi th the Executive Director 
of the Center for International Studies and the Director of the CIS 
Undergraduate Program to plan a study for which outside funding was to 
.be sought. After several general discussions we got down to the work of 
constructing a proposal to be submitted to the Institute for International 
Studies, U.S. Office of Education. The first task I reconmended that we 
assigned ourselves was an analysis of the goals and objectives which had 
been articulated for the Undergraduate Program. In addition to specific 



kinds of objectives relative to student's learning about international 
phenomena, two goals were identified which were the most intriguing to 
the three of us. 
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The first of these was: CIS interdisciplinary courses, designed for 
freshmen and sophomores, will adequately serve students as an introduction 
to the represented disciplines when compared with the standard intro- 
ductory course in these disciplines. The second goal was: The CIS courses 
will show students how knowledge is organized into disciplines, and how 
the interaction' of these disciplines can help to illuminate InternationaT 
problems. 

I found this second goal statement particularly rich in possibil- 
ities, for this goal, statement suggested a rnyriad of research questions 
growing out of the conceptual work of Schwab at Chicago and Gowin at Cornell, 
relative to the "structure of knowledge" in the disciplines. For me, the 
CIS courses presented a special environment in which to pursue the following 
kind of inquiry: given a specific discipline, what are the "key concepts" 
which permit the telling questions asked by researchers in the discipline? 
What are the methods of verification used in the discipline and how do these 
concepts, questions and methods of verification, condition the "knowledge 
claims" which result from research activities? I believed that in these 
interdisciplinary courses, with the probable confrontation of disciplines, 
^that a clarification of the attributes of the disciplinesj along the lines 
of structure of knowledge questions, would become evident. This belief was 
based in part on iriy observation of the CIS course which was in progress at 
the time of our planning for evaluation. Specifically, in the course entitled 
"Domination and Subordination", an evening session was devoted to the rise of 
German National Socialism. The political scientist, who,was the first to 



offer a short lecture tha^t evening, commented that, from the pa^t of view 
of poUtical science. Hitler's rise to power was,a pir^tteuljarly interesting 
phenomenon in that it was purely political, not direcfly relatable to other 
institutional developments in economics, religion, etc. The economist, who 
Was the second lecturer for that evening, began his remarks with a counter 
claim; namely, that one can find in the economic eonJ[|ions, direct causes 
of Hitler's rise to power. He then proceeded wi th^^^^re^red le^cture. 

The apparent contradiction of these claims iseemed attributable to 
differences between the disciplines, their key concepts,. thfi^£telling ques- 



tions, and their methods of analysis. of different kinds ofg§ata7"lf this" . 
apparent contradiction had become the subject of the evening.; session 
rather than a passing confusion for student note-takers, |*^p would have \ 
been taken in meeting the program goal of "showing how knowledge is^ organized 
into disciplines, an'd how the interaction of disci pi ines^can_illuminate \a.' 
problem". ^ - 

I have given a fairly lengthy treatment ofvmyjTnJtial infatuatilan 
with this goal for the CIS Undergraduate Program to recopstrucfe for you the 
orientation I brought to planning for an evaluation of the courses. 

My planning colleagues had another reaction:-' As^^e identified the 
difficulty in getting evidence for the achievement o/^lT^al, and as I / 



suggested, off the top of iny head, a strategy of t^^Sj^o this domain/ 

the value they placed in this "structure of knowletfge''»da*-deminished./ From 

\ / 

their point of view this goal had been written into.tfefTiterature of' the 
program Without much rigorous contemplation. Whileat irrf^ht be intefrest.ing. 
for Broek to pursue these questions, they wanted u^^'^ve alpn^n our 
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proposal writing to the central program objectives which could be more 
1/ easily assessed. 

After the meetings on CIS Undergraduate Program goals I took 
' responsibility for writing drafts of a proposal which was to be sub- 
^^mittedjo^jg Office of Education. The faculty and executive staff of 
'' 5T*3te^ comment, we would discuss and I would redraft. This process 
str^jhe^^over a number of months and for several drafts the hypothesis 
that these interdisciplinary courses would serve well as introduction to 
-the disciplines was central to the evaluation plan. 



This goal was eventually dropped from consideration because of 
tfie political environment in which the Center for International Studies 
operates. Namely, the staff of CIS was made increasingly aware by depart- 
ment chairmen that the Center's offering of courses was not well received 
by them as they were perceived to be "cutting in" on the eminence of de- 
partments as the course offering agencies of the university. Thus, to 
hold the CIS courses as alternatives to the introductory courses offered 
by departments was an impossible political position. We settled on eval- 
uation questions which were judged to be those likely to be helpful to the 
CIS program decisions and relevant to a summative statement on the sucT:Vss 
of the experimental 3-year curriculum in International Studies. 

Further modifications of questions to be asked of the Under- 
graduate Program resulted from discussions held in Washington with the 
Director and staff of the Institute for International Studies. The final 
proposal to the Institute identified the following, generally descriptive, 
questions: 
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, 1. What is occun'ng in the lecture-discussion meetings of the 

interdisciplinary courses, "Peasants, Power, and Productivity," and 
"Ethnicity, Race, and International Relations"? The various organizational 
patterns (lectures vs. sections, varieties of discussion sections, etc.), 
levels of ^student and faculty participation and interest will be among the 
phenomena to be observed and described. 

2. To what extent are there shared objectives between the courses 
to be observed? Asstiming a high degree of shared objectives despite different 
topics, to what.extent does different course organization appear to influence 



the degree to which common course objectives, are met? 

_ 3* Is the interdisciplinary approach seen as necessary and/or 
desirable ta produce understanding of complex international problems? Do 
faculty meiT4)ers concieve the purposes , of interdisciplinary education In 
international 'Studies (a) to prepare students for subsequent disciplin- 

ary work? or (b^to utilize previously gained disciplinary perspectives? Do 
faculty expectations differ from student expectations concerning the inter- 
disciplinary approach? Student and faculty members, both those associated 
and not ass^cfated with the Center for International Studies program, will 
be surveyed.^^ - 

:4. Do the attitudes of faculty and students towards the specific 
international problem area and-towards internatiorial studies in general 
change as a result of the course experience? For example, do students elect 
additional courses in international studies, reformulate their educational 
goals, etc.? 



5. What are the unanticipated outcomes of the cour 




assumed that, values, attitudes, and information not included as stated 
objectives of the courses are transmitted both to faculty and students 
as a result of their participation* ^ 

6. What guidelines for evaluation methodology for interdisciplin- 
ary International studies curricula can be suggested for further investi- 
gation? 

Generally, then, the process of selecting questions 1s charac- 
terizable in hindsight by acconmiodatlon to the realities of the interests 
of different audiences; what informati on wo uld be useful to the Institute 
for International Studies, what information would inform decisions at the 
university about the continuation of the CIS undergraduate teaching pr<^ 
gram, what information would help one course's faculty learn from the ex- 
perience of another? These distinctions which are made here were not part 
of my operating understanding of events in this planning phase as they 
took place and the consequence of my blurred vision will be clear as I 
narrate phase two: the evaluation activities. 
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We received notification of a favorable response to our proposal 
late in the Spring of 1972, at v/hich time I, as the project director, 
set out to hire a pr-incfipal investigator. It had been clear from the 
outset that v/e would be hiring an evaluation specialist to carry out , 
the study under my direction, for my responsibilities as the Associate 
Director of the Center for Improvement of Undergraduate Education 
would not permit my full attention to the project. Further, even if 
time constraints would have permitted, my professional abilities do 
not include a strong methodological background. 

I was pleased to encounter Whiton Paine, whose experience and 
expertise was directly applicable to this evaluation undertaking. 

Because of his previous commitments for the summer of 1972, 
Whiton joined the staff of our Center approximately two weeks prior to 
the beginning of the Fall semester* In this two week period he was 
responsible for the design of a questionnaire to be given to students 
enrolling in the CIS course on Rural Development, As early as this 
initial, hurried step, in the evaluation^- process our difficulties in 
communication were evident. A cause of our communications difficulty 
v/as my continuing interest in shaping the evaluation around "struc- 
ture of knowledge" issues with which he had no previous experience. 
'Every time I mentioned Schwab he looked as if I were mispronouncing 
the name of a cleaning instrument, for given his reading of the pro- 
posal to the Institute and his contacts with the staff of CIS, the 
"structure of knowledge" issues had no reference to his evaluation 
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tasks. 

As he began to gather information, my frustration increased. 
When he would return from a v/eekly fneeting of the faculty associated 
with the Rural Development course and present me with data on such 
things as the number^ of minutes they spent discussing their grading 
system, I would struggle to comprehend what on earth such data v/ere 
good for; I didn't recognize that my frustration was caused by my 
inability ^o shift my orientation to the study. What had been 
in my terms a study of interdisciplinarity was now ah evaluation of 
teaching and courses. I offer a tentative distinction regarding our 
confusion: the method of investigation which would have sought social 
science data appropriate to the "structure of knowledge" issue may 
be conceived of as educational research. The task as Whiten saw it 
was one of program evaluation with its requisite gathering of in- 
formation appropriate to a myriad of decision types. 

From the point of view of my "educational research" interests, 
the data on grading were, on the face of it, irrelevant, but from the 
point of view of the evaluation, the "information we have gathered 
across the CIS courses on grading strategies and their consequences 
may make a real contribution to future course decisions. 

We are now in the final stages of writing the summary report on 
the study and to my amazement, the Phi Delta Kappa evaluation stra- 
tegy (CIPP) which Whiton adapted for the study has yielded information 
which would inform any subsequent investigation I might pursue on the 
structure of the disciplines. From the study emerges a pattern of 
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findings relative to "integration". In one form, we find that the 
extent of "integration" of readings and lectures v/ith' other aspects 
of the course were very important to student opinion of their ex- 
perience. One of the courses, v/hich may be fairly characterized as 
a Smj6rgasbord approach to interdisciplinary teaching, with many 
lecturers each with a required reading list, v/as found unsatisfactory, 
A course which presented an analytic framev/ork which "integrated" 
the contributions of the disciplines was judged more favorably by 
students. I draw from this and from other patterns that 1n fact the 
phenomena of interest to me, namely the relationships among disci- 
plines at a level of conceptual integration, has reality in the ex- 
perience of the courses. It would be toward the urpacking of the con- 
cept of ^'ntegration" as the students used the v/ord, that I would 
first turn my attention in subsequent inquiry. 

What about other domains of effect which the collaborative evalu- 
ation study has had on me? They are many, but a few will suffice to ' 
conclude this presentation. 

The relational problems between myself, as project director, and 
Whiton, as principal investigator, were heightened by my inexperience 
in the role. I would look forward to subsequent collaborative work 
in either of two modes. 

1) Assuming that I set out to propose an evaluation project 
in which, someone was . to be brought in as the on-line, real-time agent, 
I would draw its parameters in the broadest possible fashion, bringing 
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prospective evaluation specialists into the project in an orienta- 
tion setting in which aJi significant others would participate. After 
the evaluator was selected and the project begun, my role would 1)e to 
monitor the relationships to ensure that the contracts made during 
the orientation were maintained to the conclusion of the project, 

2) The alternative approach to a clarity of role rela- 
tionship^,5 ;&o absent in my first trial, would be at the opposite 
extreme^ That is, if "educational research" were contemplated with 
the principal actor being someone other than myselfj I would establish 
the methodology in detail, specify a flow of data gathering and analy- 
sis, and establish definite reporting points so that the roles were 
clearly defined and progress of the study could be monitored carefully, 
I am prepared to believe that subsequent reality will not present 
itself in terms which can be easily treated in these two niodess but 
as distinguishable categories they will be helpful to future planning. 

Finally, I will narrate a recent experience in which reflection 
on the CIS evaluation process has had direct application. I have been 
involved recently with another interdisciplinary research and teach- 
ing program at Cornell, the Director of which has sought the colla- 
boration of our Center in the dissemination of course materials which 
they have developed over the past three years; The positive reaction 
of faculty in the CIS courses to the formative evaluation strategy 
which Whiton has employed, giving the course team data on effects of 
course components while in progress, has led me to suggest to the new 
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program that they might wish us to evaluate the courses in their pro- 
gram prior to a dissemination effort. The director of the program 
accepted this suggestion as a vehicle for objective testing of the 
course materials to supplement the subjective assessipents of the course 
developers. 

From the CIS experience I was attuned to the quality of his ac- 
ceptance of this evaluation role for our Center. The CIS evaluation 
had the enthusiastic endorsement and encouragement of the administra- 
tors of the program. In the face of faculty resistance to specific 
evaluation activities, the administrators of' CIS had been instrumental 
in negotiating reasonable constraints in which Whiton could conduct 
the evaluation. My assessment of the lower level of enthusiasm with 
which the director of the other interdisciplinary program welcomed 
evaluation has led me to -lay out a detailed procedure by which all 
faculty in his program v/ill be asked to make specific corrnii tments to 
the proposed evaluation of their courses. 

In its shortest form, the collaborative CIS project has left me 
saddened by some lost opportunities but wiser for having engaged in the 
process and for having reflected seriously on it* 
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